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ABSTRACT 

The ONAV (Onboard Navigation) 
Expert System has been developed as a real 
time console assistant for use by ONAV flight 
controllers in the Mission Control Center at the 
Johnson Space Center. This expert knowledge- 
based system is used to monitor the Space 
Shuttle onboard navigation system, detect 
faults, and advise flight operations personnel. 
This application is the first knowledge-based 
system to use both telemetry and trajectory 
data from the Mission Operations Computer 
(MOC). To arrive at this stage, from a 
prototype to real world application, the 
ONAV project has had to deal with not only AI 
issues but operating environment issues. The AI 
issues included the maturity of AI languages 
and the debugging tools, verification, and 
availability, stability and size of the expert 
pool. The environmental issues included real 
time data acquisition, hardware suitability, 
and how to achieve acceptance by users and 
management. 

Key Words: Expert Systems, CLIPS, Knowledge 
Based System Verification & Validation, 
Onboard Navigation, Mission Control Center 

1. INTRODUCTION 

The Onboard Navigator (ONAV) is a support 
room position that assists the Guidance and 
Procedures Officer (GPO) in the Mission Control 
Center. The ONAV's principal responsibilities 
are to monitor the health of the onboard state 
vector and navigational aids that are used to 
update the state vector and make 
recommendations to the GPO for actions to 
maintain the state vector. The state vector 
represents the orbiter's position and velocity at 


a given time and is used in various guidance and 
control functions on the orbiter. The state vector 
is propagated using the inertial measuring units 
(IMU's) and is updated by a variety of 
navigation sensors through a kalman filtering 
scheme. The sensors used are dependent on the 
phase of flight. ONAV supports three phases 
of flight; ascent, entry and rendezvous. During 
ascent only the IMU's are used; during entry the 
IMU's along with the tactical air command and 
navigation system (TACAN), air data 
transducer assembly (ADTA), a drag altitude 
model and the microwave scanning beam 
landing system (MSBLS) are used. Rendezvous 
uses the IMU’s, star trackers and tlie 
rendezvous radar. Basically, the ONAV job 
entails monitoring several digital displays 
that provide information on the status of the 
various navigation sensors and the onboard 
state vector. Approximately 180 parameters 
are normally monitored. The ONAV, based on 
this information, makes recommendations 
ranging from incorporating the sensor data into 
the onboard filter to recommending a ground 
derived state vector be uplinked to the onboard 
system. To be able to perform this function the 
ONAV must have an understanding of how the 
onboard software operates, how the sensors 
function, crew procedures, and a knowledge of 
kalman filtering. In addition the ONAV must 
be able to rapidly determine what information 
on the displays is important, which varies as a 
function of time, what failures have occurred, 
and what sensors are available. 

2. OBJECTIVES AND GOALS OF SYSTEM 

The objective of the system is two fold; one is to 
use the system as a console assistant and the 
other is as a training tool. Currently each 
phase of the mission requires two ONAV's for 
support; a lead and an ONAV2. The lead's 
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responsibilities are to coordinate all 
information from the console, communicate 
with the GPO and make all pertinent decisions. 
ONAV2 logs information and provides backup 
to the lead. As the system matures and both 
the GPO's and ONAV's become more confident 
with it, the goal is to eliminate the ONAV2 
position. Also the system should enhance the 
quality of mission support. This is being 
accomplished in a variety of ways. The first 
method is the automatic logging capability of 
events and recommendations with both an 
altitude and a time tag that the system 
generates. Previously the ONAV's would log 
these events by hand and if several critical 
problems occurred at once it became difficult to 
keep up. The system will eliminate this 
problem. Another enhancement is the 
incorporation of color graphics in the user 
interface. This provides an easier to read screen 
and significantly aids in problem recognition 
and resolution by providing trend analysis. 

2.1 Console Assistant 

Probably the most critical enhancement 
provided by the system lies in its ability to 
monitor all of the sensors at all times. At 
certain periods, one sensor may be more critical 
than others. As a result, the ONAV's attention 
tends to focus on the status and the functions of 
that one sensor and loses track of the others. 
The expert system, however, has the ability to 
continuously monitor all of the available 
sensors while still providing increased focus on 
the specific sensor that is providing critical 
data. Over 600 parameters are monitored by 
the entry system, 400 parameters for the ascent 
and 425 parameters for the rendezvous system. 
This function has already proven to be highly 
valuable during flight simulations conducted by 
Mission Operations Directorate (MOD), when 
the system alerted controllers to a secondary 
problem. 

2.2 ONAV Trainer 

The second objective for the system is to use it as 
a training tool. Currently, training an ONAV2 
takes over a year. An additional nine months 
is then required for ONAV1 certification. A 
major impact to the length of time that the 
training process takes is the number of 
simulation hours available. A new trainee can 


expect, at most, four hours a week in simulation 
time. The expert system will lessen the 
trainee’s dependence on simulations by 
providing the capability of running training 
simulations in an office environment. The 
biggest advantage of the use of the expert 
system as a training tool, however, lies in the 
establishment of the rulebase and its 
knowledge capture feature. When running as a 
training tool, the expert system will be 
equipped with a knowledge browser and 
desired decision rationale. This will allow the 
trainee to assimilate the information at his 
own pace and as a result he will not be as 
dependent on having an experienced person 
available to answer questions. Finally, by 
establishing a single set of agreed to rules the 
certified rulebase will provide a source of 
consistent training. 

3. IMPLEMENTATION 

The development of the rules for the ONAV 
expert system was a joint effort between the 
domain experts of the MOD and knowledge 
engineers from the Mission Support Directorate 
(MSD). The ONAV domain knowledge was 
captured through a series of meetings where 
the ONAV experts presented information to 
the knowledge engineers in a classic classroom 
setting. Initial meetings focused on the system 
functional overview. The results of the 
meetings were converted to different design 
strategies and prototypes, which could be 
critiqued by the experts. Prototype 
development then, served as the feedback 
mechanism for the meetings. During the 
development phase, when the knowledge 
engineer understood the basic operations, they 
would generally develop the agenda for the 
meetings on a particular topic and the experts 
provided the detail knowledge. The meetings 
were scheduled two to three hours once a week 
for the first three months, and after were held 
less frequently. By the end of six months the 
first prototype was completed. 

3.1 Functional Design 

It turned out that the ONAV normal task 
functions were fairly straight forward; 
however, detecting anomalies in the subsystems 
and recommending proper response required 
detailed expert knowledge. The overall system 
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design was completed in the first year of the 
development phase. This structure resulted 
from the basic nature of the ONAV task and 
from the modularity guidelines of any good 
system engineering approach. Four functional 
components of the expert system can be 
identified: (1) Fact assertion, (2) monitoring, 
(3) analysis, and (4) output. In addition, there 
is a fifth non-expert system component that is a 
part of the overall ONAV system called "data 
preparation". 

3.2 Data Prep Design 

The data preparation (data prep) component 
receives information from the operational 
environment and performs the following: 

• Collects the information required by 
the expert system 

• Performs any computations required on 
the data 

• Filters and transforms that data into a 
form suitable for the expert system. 

Often this component has been over looked, 
since this module contains no "AI" component. 
In fact, this component serves as an important 
safety net to ensure real-time (600 parameters 
every two seconds) system performance. An 
real-time monitoring system is worthless, if it 
can not keep up with the data streams. In 
addition to the real-time constraint, there is no 
reason for the expert system to examine every 
data point; especially, when the expert system 
is mostly interested in accessing the qualitative 
status of the LRUs and the overall performance 
of the entire navigation system (i.e. the 
onboard navigation Kalman filter). Thus, it is 
the data prep's responsibility to deduce the 
higher order abstractions or concepts (i.e. large 
imu biases, growing tacan residual, etc..) from a 
stream of instantaneous numerical data values. 
It terns out that the higher level abstraction 
can be modeled from a set of well defined 
mathematical and statistical functions which 
are more appropriately implemented using 
conventional programming techniques. In short, 
the data prep component converts the numeric 
instantaneous data points to a set of symbols 
which reflect the state of the environment. 
The fact assertion component simply takes the 
prepared data and puts that data required by 
the expert system into the fact base. The 
monitoring component generates intermediate 


conclusions and statuses of the individual 
subsystems that ONAV observed and manages. 
This component simply detects environment 
state changes and pass them on to the next 
component. The analysis component performs 
an overall assessment of the current status, (i.e., 
makes use of the data from the monitoring 
phase) confirms the subsystems status, and/or 
recommend actions to prevent further 
degradation of the state vector. The analysis 
phase is the heart of the knowledge base, 
because this component contains the captured 
the ONAV expertise. Specific knowledge 
implementation such as letting TACAN take 
out the state error rather than doing an uplink 
to the orbiter, or deselecting the line 
replaceable unit (LRU) when the onboard data 
bus has a commfault, were programmatically 
captured, and kept. The output component 
controls the sending of the notices and/or 
recommendations to the ONAV expert system 
console. 

3.3 Rulebase Design 

The ONAV rulebase is grouped based on four 
processing phases which is described in the 
previous section. The rule execution is governed 
by a set of phase control facts and phase control 
rules. Using this control structure, one can 
easily direct the program flow of the different 
processing modules. This approach is used 
throughout the ONAV knowledge base. For 
instance, the different navigational sensors, 
such as the IMU's, TACAN, ADTA, and MSLS 
for the Entry Nav. and the IMU's, and 
Rendezvous Radars and Star Trackers Radars 
for the Rendezvous Nav. of the Shuttle 
operation, are separated by different control 
facts. Thus the sensor modules are nicely 
grouped and the inter module rule interactions 
are kept and continuously operate in an 
opportunity data-driven fashion. In 
addition, the modular rule set enable the 
development team to support an incremental 
development process. Since, rules are changed 
quite frequently to accommodate the different 
control strategies for modeling the ONAV 
flight controllers. The modeling of the flight 
controllers' behaviors is quite unique by its own 
right. Only through the power of rulebase 
programming, one can easily encapsulate the 
set of heuristic in the form of rules. 
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3.4 Interface Design 


4. CERTIFICATION 


As for the look and feel of the system, the 
philosophy was to keep it as simple and clean 
as possible. To achieve this a pop and shoot 
method was chosen. This method consisted of a 
series of pop-up menus or toggle switches 
activated by the mouse. This provides a 
format that is easy to read and allows the 
operators to keep their eyes on the screen, a 
required feature in the highly dynamic 
environment of orbiter high speed operations. 
Messages appear in windows that can contain 
up to 20 messages and are scrollable. Status and 
quality lights take full advantage of color and 
also contain text to insure that the operator 
understands the meaning. In addition, a real- 
time plot package is developed to augmented 
and support the expert system program. For 
instance the expert system issue an off nominal 
event, due to a drifting IMU. When this 
situation is happening, the console operator can 
quickly pop up the appropriate plots to 
analysis the data. 

3.5 Data Retrieval Design 

As a real time console assistant, the expert 
system requires real time telemetry and 
trajectory data from the Mission Operations 
Computer (MOC), using a data stream called 
Generalized Data Retrieval (GDR). The GDR 
data is actually retrieved by a program called 
GDR_driver, which reads the data from the 
MOC and writes it into a shared memory 
segment. This shared memory interface is 
based on a model developed at JSC for use by 
real time applications (ref. Workstation 
Application Interface to Data Source Interface 
Agreement). This model is generic and modular 
and it provides a standard mechanism to access 
different data sources. By using this type of 
architecture, the expert system is able to 
acquire data not only from the real-time data 
source but also acquire data from a repository 
archive library. The library of test cases do not 
only sever as a way for doing software 
verification but also can be used to train flight 
controllers in their office environment. 


The certification /verification scheme devised 
for the system was established at three 
different levels. The first level consists of the 
component level; data acquisition (GDR), data 
prep, rulebase, plots and the interface. Its 
intent is to insure that the program is 
functionally correct, i.e.. that data is correctly 
gathered, that the rules fire if presented with 
appropriate actions and events, and that 
results are correctly displayed to the screen. 
The second level consists of integrated system 
testing and overall certification. The intent of 
this level of testing is to insure that the various 
constituent parts of the system function in an 
integrated manner and, more importantly, that 
the recommendations made by the system 
accurately model the expert's mental concepts 
of the state of the system and are the correct 
responses to environmental changes. The third 
level consists of an integrated workstation 
certification. This last level is accomplished 
because of the critical nature of the 
workstations and is to insure that this 
application will not interfere with any other 
workstation or mainframe applications. 
Finally, the entire process is documented and 
presented to the MOD Certification Board. 

4.1 GDR & Data Prep Certification 

For the GDR certification, data is collected 
during a sim at several time points. Hardcopies 
are requested from the operations computer 
(MOC). For each time point, data is compared 
with the MOC to the data received at the 
workstation. Once the GDR data has been 
verified, data prep module is verified and 
validated in a similar manner. The next step in 
component testing was the verification and 
validation of the rulebase, which has been the 
most difficult of the tasks. 

4.2 Rulebase Certification 

To attack the problem of verification, each 
rules is traced and checked manually with the 
experts' requirements which were decided in 
the flight technique meetings. However, the 
main problem is to devise a method to validate 
these flight procedures along with the rest of 
the system. The existing literature has little to 
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offer on verifying and validating large 
rulesbases. The final decision, after much 
debate, was the construction of an error matrix 
that contains 48 errors that a certified ONAV 
would have to be able to deal with to be 
certified. For the rulebase to be declared 
verified, the system must act correctly for each 
of the errors under several conditions. 

The method that was chosen to verify the 
rulebase has largely been dictated by the source 
of test data. The optimum method would have 
been to run a nominal case and then introduce a 
specific error in each of the following runs. Due 
to the complexities of the problem environment, 
the only source of data for the system is 
simulations or missions. However, the goal of 
integrated simulations are not gearing 
specifically to train ONAV flight controllers, 
therefore the training area as to what errors 
occur in a sim can not be controlled easily. 
Consequently the verification is somewhat of a 
hit or miss operation. Hence, the data is logged 
and evaluated when they are available, not 
necessarily in a controlled, selected fashion. 

When a sim run is selected, it is first delogged. 
In this manner actual copies of the controllers 
screens are obtained from the run. Next, an 
expert will go through the delog and determine 
what occurred in the case and what decisions 
and events the ONAV flight controller would 
have made and noted. The next step is for the 
expert to run the testcase through the expert 
system. At this time, any errors or 
discrepancies are noted. The expert then writes 
a log with the errors and discrepancies and 
what the correct action should have been. This 
is then passed to the software engineers, who in 
conjunction with the experts, work on correcting 
the problems. After the software engineers are 
done, then the corrected rulebase is passed back 
to the experts. The testcase is then replayed, 
and if all errors have been corrected and no new 
ones appear then the new rulebase is used in 
replaying all previous testcases. If everything 
checks out then the that testcase is accepted by 
all parties. 

In the event any questions arise that concern 
procedures or requirements, the action is 
referred to the ONAV working group. The 
ONAV working group was formed, with 
representatives from all the functional ONAV 


and software engineering areas, to focus the 
collective knowledge of the ONAV community 
and provide an authoritative method of 
resolving requirement conflicts or procedural 
debates. In this fashion controllers were 
provided input into the generation of the 
rulebase and a feeling that they know and 
understand what corrections have been made 
and the methodology that is incorporated in 
the rulebase. In addition the working group 
minutes serve to document the decisions made 
and provide the knowledge engineers a 
reference that they can refer to when 
creating/modifying the rulebase. 

Once all the various components have been 
verified, the system will be ready for full 
certification. At this point the system will be 
treated in the same manner as any flight 
controller for certification. To obtain final 
certification, the system is run through a full 
set of entry sims with an GPO and ONAV1 
monitoring its performance. The GPO and 
ONAV1 will then evaluate and determine 
final certification. Also during this phase the 
system will be checked to verify that it will 
not interfere with any other workstation 
application. 

Since not only expert systems but also 
workstations represent a new way of console 
operations, the system will be implemented in 
three phases. The first phase, which was used 
during rulebase verification, is for a third 
controller to monitor the system behind the 
current operator. This allows us to run the 
system during actual sims and flights without 
conflicting with console operations. Now that 
the rulebase is verified, the system has been 
moved on console and is monitored by the 
ONAV2. This phase will allow us to do the 
final interface work and answer questions such 
as, is the system easy to read and does it convey 
all the information necessary for console 
operations. Once all operators become 
comfortable with the system then at that point 
it will be ready to replace the ONAV2. 

5. STS-48 FLIGHT RESULTS 

STS-48 was the first flight using the ascent and 
entry systems as a console assistant during real 
time operations. Overall the systems 
performed very well, correctly identifying all 
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the pertinent events during ascent and entry. 
During ascent the system correctly identified 
all the major mode transitions and the events; 
events including such items as SRB SEP, roll 
reversals and MECO. Pre-launch; eight 
seconds before lift-off, navigation gets 
initialized with a state vector, the expert 
system correctly called this event. During 
flight, as previously stated, the ONAV's main 
function is to monitor the shuttle's state vector 
performance. The expert system during STS-48 
matched the ONAV's performance in 
monitoring the health of the vector. Also 
during this flight, one of the IMU's; IMU3, 
experienced a scale factor problem. The expert 
system not only identified a problem with the 
IMU but also identified the problem type. 

5.1 STS-48 Detail Description 

The expert system during entry also experienced 
good performance, even though this flight had 
some events occur that had not been seen during 
previous flights. Once again the system 
correctly identified all events and major mode 
transitions. Events during entry included sensor 
processing and TACAN lock on. During the 
STS-48 entry the runway was redesignated from 
KSC15 to EDW22, the expert system made the 
correct call in telling the GPO to change the 
ground radars to Edwards. The off-nominal 
event that occurred during this flight was that 
TACAN went into dilemma and self-test. 
Dilemma occurs when 2 TACAN LRU's are 
locked on but disagree with one another over a 
prescribed limit. The expert system correctly 
called the dilemma and self-test and also 
correctly identified which TACAN was the 
noisy unit. Proceeding through the entry 
profile; the next series of recommendations by 
the expert system consisted of calling for BFS 
transfers. The shuttle has dual software 
systems running concurrently, the primary and 
the backup. The backup state vector is not 
updated using the navigation sensors instead if 
the vector goes out of limits, a vector from the 
primary system is transfered over. The expert 
system correctly identified the BFS as out of 
limits and made the appropriate 
recommendation for a transfer and identified 
when the transfer was completed. 

One of the main functions of the ONAV; as 
stated earlier, is to determine the health of 


the nav sensors and recommend whether to use 
that information to update the nav state. After 
the TACAN dilemma had cleared and the 
noise settled down on TACAN; the expert 
system was able to reach the same conclusion as 
the operator. That conclusion was that the 
TACAN's were operating nominally and were 
safe to use. The same correct recommendation 
was also made for the Air Data system. 

The bottom line from this first flight of the 
expert system was that the rulebase contains 
the essential knowledge of the ONAV flight 
controller. The system overall did an excellent 
job of conveying this information but several 
slight changes were suggested in the interface 
to allow even easier insight into the expert 
system information. These changes included 
making some messages less verbose and 
lengthening the time between some messages so 
that the screen does not fill up as quickly. The 
fact remains however that the ONAV expert 
system is, and will become even more so, a very 
valuable tool for controllers on console. 

6. LESSONS LEARNED 

From the project development perspective, the 
number one lesson is: Even though the expert 
system contains all the "flight rules" and 
corporate flight controllers knowledge, in order 
for this new technology (expert system) to 
become available for online flight operation, 
much time and efforts have to be devoted for 
system integration and acceptance from the user 
community. 

Another interesting observation is that, time is 
one of the most critical elements on the project. 
Time provides the luxury of careful scrutiny of 
the system but if time is not carefully allocated 
it can rob the project of its momentum. The 
following problems represent areas that forced 
the project off its path and consequently slowed 
the development of the project. 

6.1 Expert Availability Issues 

As the name expert system implies, an expert is 
an integral part of developing an expert system. 
When this project first began, there were two 
ascent/entry experts and no certified 
rendezvous experts. For the projected flight 
rate at the time, six ascent/entry controllers 
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were needed and two rendezvous controllers. 
Due to the low numbers, the priorities of the 
available experts were flights, training, 
analysis and finally expert system 
development. So not only were there a limited 
number of experts, their availability was 
extremely limited also. At one point the only 
expert working on the entry system quit, 
leaving roughly a year and a half where no 
expert was available. These factors stretched 
the development out over a considerable length 
of time. This led to problems in the software 
engineering side, in that the programmers 
became restless with the lack of progress on the 
system. It consequently contributed in the 
decision by several programmers to leave the 
project. The expert system, in fact, had fallen 
prey to the very thing the system was trying to 
protect against, that of a lack of a knowledge 
base. Due to this problem, the luxury of time 
for a quick development of a prototype, at the 
start, was denied the project, consequently 
robbing the project of its needed momentum. 

Due to the drawn out development time and 
turnover problems several different groups of 
experts worked on the entry system over time. 
This meant that the rulebase represented an 
inconsistent set of methodology and at times, 
due to limited documentation, a lack of 
understanding of why things were done the way 
they were. It was in response to this, that the 
ONAV working group was formed. Its function 
is to try to reach a consensus among current 
operators and document that methodology. The 
working group concept brought a sense of 
stability into the project because it as a 
organization remained constant and left a 
documented trail. 

6.2 System Availability Issues 

Another obstacle faced by the project was the 
uncertainty over the platform that would be 
used for the final product. One of the goals of 
the project is to be used as a console assistant. 
The system is to reside on a Mission Control 
Center Upgrade (MCCU) workstation. 
Unfortunately, the expert system project and 
the MCCU project have been developed in 
parallel. During the evolution of MCCU the 
final platform went from MASSCOMP to SUN 
back to MASSCOMP, from a two MIP's to an 
eight MIP's machine and from a non-color to a 


color graphics terminals. Through it all we 
tried to keep the system as portable as possible 
and tried to get our requirements into the MCCU 
project. As a result of our philosophy of 
portability, the project did end up with some 
throw away code. The interface was done in 
four different versions; Curses, MC-Windows, 
SunView, and X-Windows versions. Even at 
this time, the MCCU's lack of robustness has 
caused the expert system setbacks in gaining 
controller confident since 90% of our sim failures 
can be attributed to workstation problems. It is 
difficult to avoid this type of problem when 
working in a high tech environment, on what 
might be called the leading edge of technology. 
The best that the development team has to be 
alert to changes and try to influence any 
changes to the environment. 

As for any system, data is the life blood of the 
system. There existed two data problems for 
the ONAV expert system, availability and 
access. As previously stated the only 
acceptable data source, due to the size of the 
problem environment, is from simulations and 
flights. This proved to be a constraint on our 
development from the start. The main problem, 
however, was in getting access to the data 
required. The system requires telemetry data 
from the orbiter, for onboard systems and state 
vector evaluations, and data from the MOC for 
output of high speed ground filter data. 
Several sources existed from which telemetry 
data could be obtained. GDR, however, was 
the only source for the ground filter. A longer 
delay in achieving access to GDR data than 
anticipated occurred due to MOC integrity 
concerns. This caused an even larger delay 
between prototype development and rulebase 
verification. Possibly if we had tried to get a 
better understanding of the system's data 
requirements closer to the start of development; 
we may have been able to piece together a set 
of nominal testcases from some analysis tools. 
This would have given us a leg up on 
verification but not solved the whole problem. 
Also from our experience with verification the 
system's data requirements can expand through 
this process. This one factor stole the most 
momentum from the project and was one of the 
most frustrating experiences to endure. The 
system was all dressed up with no place to go. 
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6.3 CLIPS Development Issues 

The C-language Integrated Production system 
(CLIPS) expert system shell was used in the 
development of the expert system. The ONAV 
project received one of first beta copies of CLIPS 
(release 3.0) for the prototype of the ONAV 
project. Even in the beta software, CLIPS 
inference engine was very robust. However, as 
the expert system development continued, more 
rules were added to the system. The rule based 
interactions became more complex. CLIPS at 
this time did not provide adequate debugging 
methods to deal with large knowledge bases, 
but in fact the state of the art for expert systems 
in general was lacking in this area. This was a 
particular problem because the expert system 
has to be embedded with other modules, such as 
the real time data acquisition, and user 
interface modules which added another layer 
of complexity. For a long time, tracking down a 
typographical error was very difficult. Since 
then, the CLIPS development team has 
developed additional tools such as the cross- 
reference, style, and verification (CRSV) tool, 
CLIPS window interfaces environments and also 
added additional syntax to the pattern 
language to deal with some of the problems. 
Finally, research is still focusing on finding the 
appropriate methodologies to perform 
verification and validation on expert systems. 
Certainly, CLIPS will also evolve along with 
the technology. 

6.4 User Community and Management Issues 

One area of particular concern was the 
acceptance of the system by users and 
management. One criticism from management 
was that the system would erode controllers 
skills, that a reliance on the system would 
eventually cause destruction of the ONAV 
knowledge base. However, our response was if 
the certification process for a controller is 
adequate then there should not be any erosion 
in skills. Users were also reluctant to use the 
system because the system did represent change 
and would require new training. Our approach 
was to bring in the users at the beginning, keep 
them involved and place the interface design 
in their control. Also all rulebase changes had 
to be approved by the expert. In these ways the 
user community became totally involved in the 


development and responsibility for the project 
thus increasing acceptance by the users. The 
major factor in acceptance, however, is simply 
time. Time for the flight controllers to use the 
system and watch as the system's performance 
proves itself worthy of confidence. Time for 
management to accept that the system will not 
be death of flight controllers as we know them. 
Time for the system to become fully integrated 
into console operations. 

7. CONCLUSION 

To arrive at this point in time, where 
users are beginning to use the system, where it is 
actually being of benefit to operators, has been, 
as chronicled in the preceding passages, a 
somewhat tortuous path. In hindsight, a 
critical element in the development of the 
project was the length of time between rulebase 
development and prototype demonstrations. 
The length of time for our project proved to be 
much longer then the ideal, causing us severe 
problems but, ultimately, not disastrous ones. 
One important lesson was to have the assurance 
of the availability of experts and a first cut of 
data before starting rulebase development. 
Without this, a rapid development of a 
realistic prototype proved impossible, which 
prevented the users from actually using the 
system. Time on the system and with the 
system, by the users, as we have experienced is 
crucial for the development of the system. The 
working group concept should also have been 
used from the start of the project to facilitate 
documentation and provide stability for the 
system. 

Time, it turned out was an enemy of this 
project. Too much was spent waiting; waiting 
for experts, waiting for workstations and 
waiting for data. As previously concluded, 
prototyping quickly, and keeping everyone 
involved is critical. Luckily, however, time 
will come to our rescue. We have spent a large 
amount of time on certification. This time has 
only increased the confidence of the controllers. 
And the more time spent in console operations, 
the greater the acceptance. 
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